A Connectionist Treatment of Grammar for Generation: Relying on Emergents

Parallel Ireatment of syntactic considerations in generation promises quality and speed. Parallelism should be used not only for simultaneous processing of several sub-parts of the output, but even within single parts. If beth types of parallelism are used with incremental generation it becomes unnecessary to build up and manipulate representations of sentence structure-the syntactic form of the output can be emergent. FIG is a structured conneetionist generator bulk in this way. Constructions and their constituents are represented in the same network which encodes world knowledge and lexical knowledge. Grammatical output results from synergy among many construetions simultaneously active at run-time. FIG incorporates new ways of handling constituency, word order and optional constituents; and simple ways to avoid the problems of instantiatien and binding. Syntactic knowledge is expressed in a simple, readable form; this representation straightforwardly defines parts of the network. 1 I n t r o d u c t i o n Generation research has not yet fully identified the advantages offered by parallelism nor the techniques necessary to take advantage of it. This is especially true for the syntactic aspects of generation. This paper presents a way to exploit parallelism for syntax in generation. The key points are: Syntactic constructions are encoded in the same knowledge network as words and concepts. Many constructions are active in parallel; there is synergy, and sometimes competition. The syntactic form of the output emerges from interactions among constructions at run t imeexplicit syntactic choice and building up of representations of syntactic structure are unnecessary. To see that this approach works for syntactically nontrivial examples, consider that FIG's outputs include: "once 1Thanks to Daniel Jurrafsky, Robert Wilensky, Dekai Wu, and Terry Regier. This research was sponsored by the Defense Advanced Research Projects Agency (DoD), monitored by the Space and Naval Warfare Systems Command under N00039-88-C-0292, and the Office of Naval Research under contract N00014-89-J-3205. An early version of this paper appears in the Proceedings of the 12th Cognitive Science Conference, Erlbaum, 1990. upon a time there lived an old man and an old woman," "one day the old man went into the hills to gather wood," " a big peach bobbed down towards an old woman f rom upstream," "an old woman gave a peach to an old man," "John broke a dish," "John made the cake vanish," and "Mary was killed;" and when producing Japanese: "mukashi mukashi aru tokoro ni ojiisan to obaasan ga sunde imashita," "aru hi ojiisan wa yama e shibakari ni ikimashita," "kawakami kara ookii momo ga donburiko donburako to obaasan e nagarete kimashita," "ojiisan wa meeri ni momo o agemashita," and "meeri o koroshimashita." Section 2 discusses parallelism in syntax and presents the basic proposal. Section 3 presents a framework for connectionist generation, and Section 4 elaborates the proposal in this framework. Sections 5 through 8 discuss an implementation of these ideas: Section 5 presents a representation for grammatical knowledge, Section 6 explains how the proposal accounts for specific syntactic phenomena, Section 7 presents an example of the generator in action, and Section 8 discusses general implementation issues. Section 9 summarizes. 2 P a r a l l e l S y n t a x This section discusses two types of parallelism for syntax, proposes that a generator should have both of them, and sketches out the advantages of such an approach. Natural language generation research traditionally assumed that syntactic choices are made in a fixed (and generally top-down) order. Yet, for incremental generation at least, it is clear that a fixed order of decisions is not appropriate. This realization has led to generators which work on several parts of the input in parallel, simultaneously building several sub-trees. Recent work in this area includes (De Smedt 1990) and (Finkler & Neumann 1989). I will refer to this type of parallelism as 'part-wise' parallelism. A second kind of parallelism involves using several constructions to generate even one part of the output. As far as I know, this 'within-part' parallelism has not been proposed in the generation literature. It has proven useful in linguistics. In Fillmore's Construction Grammar the syntactic